Discovery of Latent Factors in High-dimensional Data Using Tensor Methods

OF THE DISSERTATIONDiscovery of Latent Factors in High-dimensional Data Using Tensor MethodsByFurong HuangDoctor of Philosophy in Electrical and Computer EngineeringUniversity of California, Irvine, 2016Assistant Professor Animashree Anandkumar, Chair Unsupervised learning aims at the discovery of hidden structure that drives the observationsin the real world. It is essential for success in modern machine learning and artificial intel-ligence. Latent variable models are versatile in unsupervised learning and have applicationsin almost every domain, e.g., social network analysis, natural language processing, computervision and computational biology. Training latent variable models is challenging due to thenon-convexity of the likelihood objective function. An alternative method is based on thespectral decomposition of low order moment matrices and tensors. This versatile frameworkis guaranteed to estimate the correct model consistently. My thesis spans both theoreticalanalysis of tensor decomposition framework and practical implementation of various appli-cations. This thesis presents theoretical results on convergence to globally optimal solution of tensordecomposition using the stochastic gradient descent, despite non-convexity of the objective.This is the first work that gives global convergence guarantees for the stochastic gradientdescent on non-convex functions with exponentially many local minima and saddle points. This thesis also presents large-scale deployment of spectral methods (matrix and tensordecomposition) carried out on CPU, GPU and Spark platforms. Dimensionality reductiontechniques such as random projection are incorporated for a highly parallel and scalablexvi tensor decomposition algorithm. We obtain a gain in both accuracies and in running timesby several orders of magnitude compared to the state-of-art variational methods. To solve real world problems, more advanced models and learning algorithms are proposed.After introducing tensor decomposition framework under latent Dirichlet allocation (LDA)model, this thesis discusses generalization of LDA model to mixed membership stochasticblock model for learning hidden user commonalities or communities in social network, con-volutional dictionary model for learning phrase templates and word-sequence embeddings,hierarchical tensor decomposition and latent tree structure model for learning disease hierar-chy in healthcare analytics, and spatial point process mixture model for detecting cell typesin neuroscience.